AUTOMATIC CONTINUITY FOR THE UNITARY GROUP 



TODOR TSANKOV 



Abstract. We show that every homomorphism from the infinite-dimensional 
unitary or orthogonal group to a separable group is continuous. 



1. Introduction 

Many uncountable groups come naturally equipped with a group topology that 
greatly facilitates their study. When dealing with topological groups, one can use 
powerful tools such as Haar measure, Lie group theory, Baire category methods, etc. 
On the other hand, if one wants to study the groups abstractly, far fewer tools are 
available. One way to approach this problem is via reconstruction theorems that 
recover the topology of the group from its algebraic structure. Perhaps the strongest 
results in the literature of this type have the following form: let G be some Polish 
group, then every homomorphism from G to a separable group is continuous. (One 
always needs some restriction on the target group to avoid the trivial example of the 
identity map from G to itself equipped with the discrete topology. The restriction 
of separability is rather mild and does not exclude most interesting examples.) In 
that case, we will say that G has the automatic continuity property. 

The first automatic continuity results of this type were obtained by Kechris- 
Rosendal [KR , using the techniques of ample generics developed by Hodges, Hod- 
kinson, Lascar, and Shelah [HHLS . Soon thereafter other results in the same vein 
followed: Rosendal-Solecki [K5], Rosendal QH], Kittrell-Tsankov [KTj ; see the 
recent survey |R2| for more details. 

The following is the main theorem of this paper. It answers a question of 
Rosendal IR1I. 



Theorem 1. The unitary (orthogonal) group of an infinite- dimensional, separable, 
complex (real) Hilbert space, equipped with the strong operator topology, has the 
automatic continuity property. 

The proof of this theorem relies in an essential way on recent work of Ben Yaacov- 
Berenstein-Melleray [BYBMj . who, extending the approach of jKRj . developed a 
new theory of topometric groups with ample generics that we briefly describe. 
They observed that Polish groups that are naturally presented as isometry groups 
of metric spaces, apart from the Polish topology of pointwise convergence, also 
carry a natural bi-invariant uniform structure, namely, the one defined by uniform 
convergence on the metric space. For example, for the unitary group of an infinite- 
dimensional Hilbert space, the Polish topology is the strong operator topology and 
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the bi-invariant uniform structure is given by the operator norm. One of the main 
results of BYBM is that if such a group G has ample topometric generics, then 
any homomorphism from G to a separable group that is continuous in the uniform 
topology is also continuous in the Polish topology. They also found examples of 
groups with this property; the ones that will be important for us are the unitary 
group and the automorphism group of a Lebesgue probability space Aut(X, /i). 
Using this theorem and previous work of Kittrell-Tsankov [ KT] , they were able to 
show that Aut(A, /i) has the automatic continuity property. 

To avoid repetition, in the remarks below we concentrate on the unitary group 
but they are also valid for the orthogonal group. 

One immediate corollary of our theorem is that every action of the unitary group 
U(H) by homeomorphisms on a compact metrizable space or by isometries on a sep- 
arable metric space is automatically continuous. Thus, by Gromov-Milman jGMj . 
every action of U (H) on a compact metrizable space has a fixed point and Kir- 
illov's and Olshanski's classification K3|[0] of continuous unitary representations 
of U(H) becomes a classification of all representations of the discrete group U{H) 
on a separable Hilbert space, etc. 

Combining the theorem with the result of Stojanov [S] that the unitary group 
is totally minimal (i.e. every continuous homomorphism to a Hausdorff topological 
group is open), we obtain the following corollary. 

Corollary 2. Let G be the unitary or the orthogonal group. Then the following 
hold: 

(i) G admits a unique separable group topology; 

(ii) if G' is a Polish group, and <f>: G — > G' a homomorphism, then <j>{G) is a 
closed subgroup of G' . 

Atim A had previously shown that the unitary and orthogonal groups admit a 
unique Polish group topology. 

This corollary rules out the existence of non-trivial homomorphisms from U(H) 
to Polish locally compact groups, or more generally, Polish groups admitting a left 
invariant complete metric, or Polish totally disconnected groups, etc. 

Another corollary of the theorem is that the quotient of the unitary group by the 
normal subgroup of unitary operators that differ from the identity by a compact 
operator does not admit a non-trivial homomorphism to a separable group. This 
generalizes a result of Pickrell [P] who had shown that this group does not admit 
continuous (with respect to the quotient of the norm topology) non-trivial unitary 
representations on a separable Hilbert space. 

Theorem [T] should be contrasted with the situation for finite-dimensional unitary 
groups: it is a result of Kallman [Klj . and independently Thomas [T], that GL(n, C) 
embeds as a subgroup of Soo and as U(n) is connected, the restriction of this 
embedding to U (n) cannot be continuous. 

Remark. In |Rlj . Rosendal asks whether there is an (infinite) compact metrizable 
group that satisfies the automatic continuity property. It follows from the Peter- 
Weyl theorem and the result mentioned above that every compact metrizable group 
embeds in Soo and therefore if such a group satisfies the automatic continuity 
property, it must be profinitc. 
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2. Proofs 

We set to prove Theorem [1] and first concentrate on the complex case. Let H 
be an infinite-dimensional, separable, complex Hilbert space and U (H) its unitary 
group. 

A subset W of a group is called symmetric if W = W' 1 . A symmetric set is 
called countably syndetic if countably many left (or, equivalently, right) translates 
of it cover the group. By [RS| Proposition 2], to establish Theorem [TJ it suffices to 
prove the following. 

Theorem 3. If W is a symmetric, countably syndetic subset ofU(H), then W 506 
contains an open (in the strong operator topology) neighborhood of the identity. 

By the results of [BYBM , it suffices to find a subset of some fixed power of W 
that is open in the norm topology, and this is what we do below. As we are going 
to use automatic continuity results for various subgroups of U(H), we start with 
the following preliminary lemma. 

Lemma 4. Suppose that G is a group, G is a subgroup and W is a countably 
syndetic set for G. Then G' (~l W 2 is countably syndetic for G' . 

Proof. Let G = \J n g n W and A = {n £ N : g n W flG' / 0}. For every n £ A, 
choose h n £ g n W fl G', so that h n = g n w n with w n G W. Let now ft e G' be 
arbitrary. There exists n £ A and w £ W such that h = g n w. Then h = h n w~ l w £ 
h n (W 2 r\G'), showing that G' = \J neA h n {W 2 n G'). □ 

We establish some notation. If if is a closed subspace of H, denote by U(K) 
the unitary group of K and by Gk the pointwise stabilizer of K in U(H): 

Gk = {u £ U(H) : ux = x for all x £ K}, 

so that U{K) is naturally isomorphic to G K ±. Denote by Ik the setwise stabilizer 
of K in U (H) and note that Ik — Gk X G K x ■ Define Pk ■ Ik ^ G K ± as follows: 
if u = u\U2 £ Ik with u\ £ Gk and ui £ G k j-, let Pru — ui- If if is a subspace 
of H, say that K is balanced if both K and K are infinite-dimensional. 

Fix now a countably syndetic W C U(H). Then there exists a countable subset 
{s n : n £ N} C U(H) such that U{H) = \J n s n W. 

Say that a set A C U(H) is full for a subspace K C H if for every u £ U(K), 
there exists v £ An Ik such that v\k = u. The following diagonalisation argument 
is quite standard and was first employed in a similar setting in [RS . 

Lemma 5. Suppose that H = (§) n K n , where each K n is infinite- dimensional. 
Then there exists n such that W 2 is full for K n . 

Proof. It suffices to see that some s n W is full for K n because then W 2 = (s n W)~ 1 (s n W) 
would be full as well. If not, then for each n, there exists u n £ U(K n ) such that 
for all u £ s n W that leave K n invariant, u\k„ u n . Then Q) k Uk G G but 
Q) k Uk ^ s n W for all n, contradiction. □ 

The proof of following lemma is similar to the one of |KT| Lemma 3.3]. 
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Lemma 6. Suppose that H = K n , where each K n is infinite- dimensional. 
Then there exists n such that G K ± C W 2A . 

Proof. A unitary operator u £ U(H) is called a symmetry if u = 1, i.e. there exists 
a decomposition ff = iii ® i?2 such that ux = x for all a; £ i?i and ux = —a: for all 
x G i?2- Note that every two symmetries for which the corresponding eigenspaces 
H\ and H2 are infinite-dimensional are conjugate in U(H). Halmos-Kakutani [HK 
have shown that every unitary operator is the product of four symmetries; even 
though they do not mention it explicitly, it follows from their proof that one can 
choose the symmetries so that they have infinite-dimensional eigenspaces. 

Let now K n be such that W 2 is full for K n as given by Lemma OH Let e%, e^, . . . 
be an orthonormal basis of K n and let {Ai : i £ 2 N °} be a family of subsets of 
N such that Ai 1 AAi 2 is infinite and co-infinite if i\ ^12- Let v% £ U(K n ) be the 
symmetry defined by 

\ej, \ij£A l \ 
[-ej, iij<£Ai. 

and let Ui — v% © l^u. £ U(H). By the pigeonhole principle, there exist i\ 7^ ii 
and n such that Ui l: Ui 2 £ s n W. Then u^u^ — uZ tti a £ W 2 and by the choice 
of the sets A», v^v^ is a symmetry of i4T ra with infinite-dimensional eigenspaces. 
Now applying the result of [HKj , we obtain that every element of G K x is a product 
of four conjugates of u~ 1 Ui 2 and thus, by the fullness of W 2 for K n , Gx± C 

{w 2 w 2 w 2 ) A = w 2A . 1 71 □ 

Let {X,n) be a standard probability space and denote by Aut(X, fi) the group 
of all measure-preserving automorphisms of (X,/i). We equip it with the weak 
topology, which is the coarsest topology that makes the maps AutpT, fi) R, 
T i-> fi(T(A)AB), where A and B are measurable subsets of X, continuous. We 
also consider the uniform distance d on Aut(X, /i) defined by 

d(Tx,T 2 ) = ^({x £ X : Tix ^ T 2 a;}). 

Note that the topology defined by d is strictly finer than the weak topology. 

Lemma 7. Let V C Aut(X,^) be a countably syndetic subset of Aut(X, fj,). Then 
V contains a weakly open neighborhood of the identity. 

Proof. This is a combination of results of [KT] and jBYBMj . By [EH Theorem 3.1], 
V 38 contains a d-ball B e of radius e around the identity in Aut(X, (j) for some 
e > 0. (Even though the paper jKTj deals with full groups of equivalence relations, 
this proof works equally well for the entire group Aut(X, //). See also [BYBM ( 
Appendix A] for an explicit rendering of the argument.) Now applying the fact that 
Aut(X, fi) has ample topometric generics (see |BYBM1 Section 5.4]) and jBYBM, 
Theorem 4.4], we obtain that V 38 V 10 = V 48 contains a weak neighborhood of the 
identity. □ 

We now return to the unitary group. 

Lemma 8. There exists a balanced subspace K of H such that Gk Q W 2A and 
W 120 contains an open neighborhood of the identity in Ik- 

Proof. Let H = K n be a decomposition of H into infinite-dimensional sub- 
spaces and let K n be the subspace given by Lemma [6] Put K = , so that 
G K C W 2A . 
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Consider the probability space (X, /j) = (R N , z^ N ), where v denotes the standard 
Gaussian measure. Then there exists an embedding 7: U(K) — > Aut(X, fi) which 
has the following property: if n: Aut(X, /1) — > U(L 2 (X,/i)) denotes the standard 
embedding, then there exists a balanced subspace K' C L 2 {X,p) and a unitary 
isomorphism $ : K — > K' such that 



This is the so called Gaussian construction; see, for example, |K2| Appendix E] for 
details. 

Identify H with L 2 (X, p) via an isomorphism which sends K to K' and whose 
restriction to K is equal to We thus obtain embeddings 



such that if we put 9 = n o 7, then K is invariant under 0{U{K)) and for every 
ueU(K), 6{u)\ K =u. 

Equip the three groups U(K), Aut(X,/i) and U(H) with their Polish topolo- 
gies and observe that both 7 and K are homeomorphic embeddings. By Lemma 21 
W 2 n re(Aut(X, p)) is countably syndetic in /c(Aut(X,/z)). By Lemma [3 (PF 2 D 
fc(Aut(X, y^))) 48 C PF 96 contains an open neighborhood of the identity in /t(Aut(X, /i,)), 
and in particular, there exists an open neighborhood of the identity O in 9{U{K)) 
such that O C VF 96 . 

Let t be the natural isomorphism U (K) — > G K ± given by t(u) — u © l^a. and 
let p = 9 o r _1 , so that p is an isomorphism G#o. — > 9(U(K)). We also note that 
for every u G Grx, Pk(p(u)) — u. Let O' = /0 _1 (O) C Gf^x. We now show that 
O'Gk Q W 120 and as O'Gk is open in Gjci x Gk = this will complete the 
proof. Let (ux,u 2 ) G O' x G#. We have 



Lemma 9. Lei K and L be two infinite- dimensional subspaces of a Hilbert space 
H . Then there exist two infinite- dimensional subspaces K 1 C K and L' C L such 
that K' ± L'. 

Proof. We build inductively two orthonormal sequences ex, e%, . . . G K and fx, fi, ■ ■ ■ G 
L such that (ef, fj) — for all i,j. First pick a unit vector ei € K arbitrar- 
ily. Assuming that ex, /1, e%, fi, . . . , e n have been constructed, choose /„ G L and 
e n+ \ G K so that /„ is perpendicular to ex, fx, ■ ■ ■ , e« and e„ + i is perpendicular to 
ex, fx, ■ ■ ■ j e n , /„ (this can be done because both if and L are infinite-dimensional). 
Finally, let K' = spanfe^}, L' = span{/i}. □ 

For the next lemma, recall that Soq is the Polish group of all permutations of an 
infinite, countable, discrete set equipped with the pointwise convergence topology. 

Lemma 10. Let K be a subspace of H that satisfies the conclusion of Lemma 
and L C H be infinite- dimensional. Then there exists v G W &A and V C L such 
that vL' = . 



(k °j)(u)\k' = <&u$> 1 for every u G U(K). 



U(K) ^ Aut(X, p) ^ U(H) 



uxu 2 = UxP K ±{p(ui))P K ±(p{ux)) 1 u 2 

= P K {p{ux))P K x (p(ux))P K ± (piux))- 1 ^ 




□ 
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Proof. By Lemma |9j there exist infinite-dimensional K\ C K and L\ C L such 
that K\ _L L\. Let Bi and i?i be orthonormal bases of K\ and L\ respectively 
and consider the embedding 6\ : Soo —> G(k 1 (bl 1 ) ± given by the action of Soo on 
i^iSii by permuting the basis B t U B[ . Let Qi = 0i(5oo)- By Lemma |H Vy 2 nQi 
is countably syndetic for Qi, so by the proof of |KRI Theorem 6.24] (see also }RSj . 
where this fact is mentioned explicitly), (W 2 H Qi) 10 C T4 720 contains an open 
subgroup V\ of Q\. Then there exist an element v\ € V\ and infinite-dimensional 
subspaces K2 Q K\ and L2 C Li such that 1*1X2 = -^2- 

Let now B 2 ,B' 2 , B 2 be orthonormal bases of K 2 , K Q K 2 , and K, respectively. 
Let 8 2 : S'oo - ► U(H) be the embedding given by the action of S'oo on H by permut- 
ing the basis B 2 U B' 2 U Z? 2 ' and let Q2 = 8 2 (Soo). Then, as above, VK 20 contains an 
open subgroup V2 of Q 2 . A basic open subgroup of Q 2 is the stabilizer of finitely 
many elements of the basis B 2 U B 2 U B 2 ' ; suppose that V2 is the stabilizer (in Q 2 ) 
of A U A' U A", where A, A', A" are finite subsets of B 2 ,B' 2 , B' 2 \ respectively. As B 2 
is infinite, there exists v 2 € Gk^Qi such that V2(B 2 ) 3 j4UA'. Finally, let V3 € V2 
be such that v 3 (v2 (B 2 )) = B 2 U B' 2 , so that ^3(^2 (-^2)) = K 1 -. (We can achieve 
this as follows: let C" U C 2 be any splitting of B 2 \ A" into two infinite pieces; then 
define v$ to be a permutation of B 2 U B 2 U B' 2 that fixes A U A' U A" and sends 
B 2 ' \ A" to C'{, v 2 {B 2 ) \ (A U A 1 ) to (B 2 U B 2 ) \ (A U A'), and (B 2 U B 2 ) \ t; 2 (B 2 ) 
to C'l) 

We finally have that V3V2V1L2 = K 1 - and 

viV2V X e V 2 G K Vi c VF 20 W^ 24 VF 20 = VF 64 , 
completing the proof of the lemma. □ 

If r > 0, denote by B r the open ball of radius r around the identity in U(H) in 
the operator norm. 

Lemma 11. There exists e > such that B e C W 496 . 

Proof. Suppose K C H is as in the conclusion of Lemma |S] and let e > be such 
that B e DI K C W 120 . We will now check that B e C VK 496 . Let «eB e and let L be 
a balanced subspace of H invariant under u (which exists by the spectral theorem). 
Let Mi = Pjju, and u 2 = Pl± u so that u = U1U2 and note that Ui,u 2 G B e . By 
Lemma [TU1 there exists V C L 4 - and vi S PF 64 such that i>iL' = if ; in particular, 
viuiv^ 1 G G/fj- C As the norm is invariant under conjugation, we have that 
vmxVi 1 e B t C\I K Q W 120 , whence u x G VF 64 VF 120 VF 64 = VF 248 . A similar 
argument shows that 112 <E W 248 , so w S VK 496 and we are done. □ 

Now it is easy to complete the proof of Theorem [3J If A C U(H), denote 

(A) t = {ue U{H) :3a e A \\a- u\\ < e}. 

Let e be the one given by LemmaQH By BYBM, Theorem 4.4], (W w ) e contains an 
open (in the strong operator topology) neighborhood of the identity but {W m ) t = 
W w B e C W 506 . 

We now indicate the necessary changes in the proof to establish the theorem in 
the case of a real Hilbert space H . As the proof contains practically no analytic 
arguments, these changes are rather minor. We replace the occasional use of the 
spectral theorem by the following standard lemma. 
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Lemma 12. Let H be a real, infinite- dimensional Hilbert space and T an orthogonal 
operator. Then there exists a decomposition H = ©„ eN H n such that each H n is 
infinite- dimensional and invariant under T. 

Proof. Let He = H (g> C be the complexification of H and let T c be the complex- 
ification of T so that T c G U(Hc). There is a natural operation of conjugation 
on He , £ 8> z n- £ <£> one can identify if with the real subspace of H c given by 
{77 e i/ c : 77 = rj}, T c commutes with conjugation, and T c \h = T. By the spec- 
tral theorem, H c decomposes as a sum @ n K n of T c -invariant infinite-dimensional 
subspaces and by rearranging, we can further assume that K n — K n for each n. 
Then for each n, H n = {rj + rj : rj G K n } is an infinite-dimensional subspace of H, 
and the H n s are pairwise orthogonal and invariant under T. □ 

We now go through the lemmas above one by one. The proof of Lemma [5] goes 
through verbatim in the real case. The proof of Lemma |H] goes through verbatim 
as well; only in the proof of the Halmos-Kakutani theorem, we need to replace 
the use of the spectral theorem by Lemma [T2j Lemmas I8I9I10I survive without 
changes (we note that the Gaussian construction works equally well, and in a sense, 
even more easily, in the real case) . In Lemma 111! we need once again to invoke 
Lemma [T21 Finally, to complete the proof, in order to apply [BYBM, Theorem 4.4], 
we need to check that the orthogonal group has ample topometric generics. To 
verify this for the unitary group, the authors of |BYBMj use a result of Rosendal 
( |R3[ Proposition 6.6]). As the author of |R3j notes explicitly, his proof works 
equally well for the orthogonal group. 

Proof of Corollary^ (|TJ) . Let r denote the natural Polish topology on G and r' 
be some other separable topology. Then the identity map (G, r) — > (G,t') is 
continuous by Theorem [I] and open by the theorem of Stojanov [Si and therefore a 
homeomorphism. 

(|n|) . If G' is Polish and <f>: G — > G' is a homomorphism, then <j) is continuous by 
Theorem[T]and open by (SJ. Then 0(G) is isomorphic (as a topological group) with 
G/ker0, therefore Polish, whence closed in G'. □ 
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